Kernel CCA for multi-view learning of acoustic features using articulatory measurements

We consider the problem of learning transformations of acoustic feature vectors for phonetic frame classification, in a multi-view setting where articulatory measurements are available at training time but not at test time. Canonical correlation analysis (CCA) has previously been used to learn linear transformations of the acoustic features that are maximally correlated with articulatory measurements. Here, we learn nonlinear transformations of the acoustics using kernel canonical correlation analysis (KCCA). We present an incremental SVD approach that makes the KCCA computations feasible for typical speech data set sizes. In phonetic frame classification experiments on data drawn from the University of Wisconsin X-ray Microbeam Database, we find that KCCA provides consistent improvements over linear CCA, as well as over single-view unsupervised dimensionality reduction.